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The  Decline  and  Fall  of  Joint  Acquisition  Programs 

Findings  from  Prior  Joint  Program  Research 

•  “Joint  programs,  whether  large  or  small,  in  development  or  production,  and  irrespective  of 
age,  are  statistically  more  likely  to  encounter  programmatic  breaches  than  their  single 
system  counterparts.” 

—  “The  Acquisition  of  Joint  Programs:  The  Implications  of  Interdependencies  ”  by  Brown,  Flowe,  Hamel 

•  “The  resistance  to  bi-service  programs  is  thus  deep-seated  and  powerful...  When  asked 
to  consolidate  programs  they  will  make  powerful  arguments  in  favor  of  the  attributes  of 
their  own  proposal  and  against  the  attributes  of  the  other  service’s  proposal.” 

—  “Inter-Service  Weapons  Rivalry  ”  by  Robert  F.  Coulam 

•  “The  common  problems  that  cause  delays  and  cost  overruns  can  typically  be  attributed  to 
‘mission  creep’  requirements  that  surface  during  the  SDD  phase.” 

—  “Program  Management  Challenges  in  a  Joint  Service  Environment”  by  LtCol  Dorothy  E.  Taneyhill 

•  “...Service  parochialism...  conspires  to  ensure  that  individual  Service  equities...  dominate 
the  acquisition  process.” 

—  “Reforming  the  Joint  Acquisition  Process  ”  by  Brinson,  Jones,  and  Kelly 

•  “Pentagon  decisionmaking  reforms  since  World  War  II  are  largely  a  history  of  efforts  to 
curtail  the  power  of  the  Services  to  veto  joint  solutions  that  serve  the  entire  military 
better.” 

—  “Reforming  Pentagon  Strategic  Decisionmaking”  by  Lamb,  Lachow 
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The  Decline  and  Fall  of  Joint  Acquisition  Programs 

Joint  Programs 


1.  A  JPO  PM  has  six 
stakeholder  programs 
planning  to  use  their 
joint  infrastructure 
software... 


2.  ...but  each  program 
demands  at  least  one 
major  feature  be  added 
to  the  software  just 
for  them. 


7.  As  cost  escalates 
and  schedules  lengthen, 
participation  in  the 
joint  program  unravels 
and  collapses. 


6.  With  one  stakeholder 
gone,  the  amortized  costs 
for  the  other  programs 
increase  further — and 
another  program  leaves. 


3.  The  JPO  agrees  to  the 
additional  requirements,  for 
fear  of  losing  stakeholders 
(who  could  build  custom  software) 


5.  As  the  schedule 
slips,  one  program 
decides  to  leave  the 
joint  program  and 
develop  its  own 
custom  software. 


This  scenario  aggregates 
three  SEI  software-reliant 
system  acquisition  ITAs 
conducted  in  2006-2009. 


4.  The  additional  design 
changes  and  coding 
significantly  increase 
total  cost,  schedule, 
complexity,  and  risk. 
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What  are  Social  Dilemmas? 

What  if  we  all  could  be  better  off,  but  no  one  has  an  incentive  to  change? 
Dilemmas  are  all  about  cooperation — and  there  are  two  basic  types: 

Social  Trap:  “The  Tragedy  of  the  Commons ” 

•  Someone  wants  a  benefit  that  will  cost  everyone  else 

•  Some  are  tempted  by  that  benefit,  but  if  all  do,  everyone  is  worse  off. 

Social  Fence:  “Producing  a  Public  Good ” 

•  Someone  faces  a  near-term  cost  that  would  benefit  everyone  else 

•  Some  try  to  avoid  the  cost,  but  if  all  do,  everyone  is  worse  off. 

Social  Trap  Examples: 

Overfishing :  Everyone  catching  more  fish  will  mean  there  are  no  more  fish 
Congestion :  Everyone  using  a  car  for  their  convenience  creates  traffic  jams 
Pollution:  It’s  cheaper  to  pollute,  but  everyone  else  pays  the  price  in  smog 

“Individually  optimal  decisions  lead  to  collectively  inferior  solutions” 


^ _ 

The  “Tragedy  of  the  Commons”  is  a  multi-player  version  of  the  “Prisoners’  Dilemma” 
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The  Decline  and  Fall  of  Joint  Acquisition  Programs 

What  Key  Mechanisms  Undermine  Joint  Programs? 

Stakeholders  of  joint  programs  demand  additional  (custom) 
requirements  later  in  the  lifecycle 

•  The  more  time  that  elapses  between  establishing  the  baseline  requirements 
and  introducing  new  requirements,  the  more  effort  it  takes  to  develop  the 
software  for  those  requirements. 

Underbidding  is  used  to  win  development  contracts 

•  Underbidding  leads  to  schedule  pressure  that  can  shortcut  quality  processes 
and  increase  firefighting  and  staff  burnout. 

Developers  are  motivated  to  show  good  progress  early  in 
development 

•  Failing  to  address  the  hardest  requirements  early  in  the  development  lifecycle 
(such  as  in  the  first  increment)  in  order  to  show  better  progress,  leads  to 
greater  rework  to  address  them  later  in  the  lifecycle,  slowing  progress. 


Take-away 

Understanding  the  underlying  mechanism  that  drives  joint  program  collapse  is  essential  to  fixing  it 
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The  Decline  and  Fall  of  Joint  Acquisition  Programs 

System  Dynamics  Simulation  Model  Overview 


The  Decline  and  Fall  of  Joint  Acquisition  Programs 

Grounding  and  Validating  the  Model 

Model  Validation  Approaches: 

1)  Historical  Joint  Program  Performance  Data 

•  Data  tracking  the  monthly  progress  was  obtained  from  the  PMO  and  contractor,  and 
correlated  with  a  timeline  of  other  events/actions/decisions 

2)  Joint  Acquisition  Program  Workshops 

•  Conducted  two  2-day  workshops,  each  held  with  senior  experts  from  a  single  joint  program 

•  Human  subject  research  protocol  ensured  confidentiality  of  the  data  collected 

3)  Game-based  Joint  Program  Experiment 

•  Some  “less  attractive”  behaviors  of  joint  program  stakeholders  aren’t  willingly  revealed 

•  Developing  web-based  game  engine  to  host  controlled  experiments  with  experienced 
acquisition  staff,  to  collect  data  on  realistic  behaviors  in  joint  acquisition  scenarios 

Collected  Quantitative  and  Qualitative  Data  Corrects  and  Refines  the  Initial  Model 

•  Refine  joint  program  model  structure  based  on  the  empirical  stakeholder  behaviors 

•  Use  collected  performance  data  to  drive  selected  joint  program  model  inputs 

•  Compare  model  simulation  performance  to  historical  program  performance 


Take-away 

It’s  not  enough  to  have  only  a  notional  or  qualitative  understanding  of  joint  program  dynamic  behaviors. 

2014  AFCEA 

Acquisition  Research  Symposium 
15  May  2014 

©2014  Carnegie  Mellon  University 


Software  Engineering  Institute  CarnegieMellon 


8 


The  Decline  and  Fall  of  Joint  Acquisition  Programs 


Key  Research  Findings 

A  key  tipping  point  is  crossed  due  to 
unrealistic  schedules/late  requirements 

•  Developer  productivity  as  a  function  of 
schedule  realism  &  late-addition  req’ts 

•  Program  managers  need  to  know  if  they’re 
about  to  “fall  off  a  cliff”  at  a  tipping  point 

Stakeholders  played  “Schedule  Chicken” 

•  Stakeholder  platforms  hid  their  own 
performance  problems  behind  the  joint 
program’s  schedule  issues 


Developer  Productivity 

(normalized1) 


1-1.2 
0.8-1 
0.6-0  8 
0. 4-0.6 
10.2-0.4 
10-0.2 


High 


Medium 

Schedule 
Low  Realism 


Level  of  Custom  Req’ts 
Added  After  Baseline 


There  is  an  inherent  “social  trap”  at  the  heart  of  a  joint  program 

•  The  JPO  accepting  more  late  custom  requirements  to  keep  stakeholder  programs 
engaged  (to  prevent  defection/collapse)  is  a  “no  win”  social  trap  that  leads  to  program 
collapse,  especially  when  an  unrealistic  schedule  exacerbates  the  expanded  scope. 

Mandated  joint  program  participation  can  have  severe  unintended  consequences 

•  If  a  stakeholder  program  is  unwillingly  forced  to  join  a  joint  program,  they  will  find  a  way 
to  leave — and  their  defection  will  likely  lead  to  a  cascade  of  defections  that  will  collapse 
the  joint  program 
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Candidate  Social  Dilemma  Solutions 

Goal:  Decrease  custom  requirements  demanded  by  stakeholder  programs 
Goal:  Increase  realism  of  development  schedule 


Altruistic  Punishment 


PRO:  Pay  to 
penalize 
uncooperative 
partners 
CON:  May 
escalate  and 
cause  retaliation 


Privatization 


•  PRO:  Gives 
incentive  to 
care  for  what’s 
yours 

•  CON:  Loses 
reason  for 
cooperation 


Shared  Destiny 


PRO:  Incentivize 
based  on 
program 
outcome 
CON:  Incentive 
may  be  far  in  the 
future 


PRO:  Regulates 
the  good, 
prevents  overuse 
CON:  Unpopular 
to  enforce  a 
mandate 
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The  Decline  and  Fall  of  Joint  Acquisition  Programs 

Conclusions 

Current  Status 

•  Preliminary  observations  show  a  tipping  point  toward  much  lower  overall 
development  productivity  contributing  to  the  joint  program  social  dilemma 

Potential  Areas  of  Future  Research 

•  Model  possible  solution  approaches  and  analyze  their  effectiveness 

-  Identify  the  most  promising  solution(s)  to  be  piloted  on  real  programs 

•  Forecast  acquisition  program  performance 

-  Use  a  parameterized  model  as  a  management  decision  support  tool 

-  Run  “what  if?”  scenarios  on  acquisition  programs  in  progress  &  compare  outcomes 

-  Generate  3-D  maps  of  the  decision  space  to  help  decision-makers  navigate  it 

•  Influence  future  policy  for  joint  acquisition  programs 

-  Help  shape  how  joint  acquisition  programs  are  conducted  through  policy  changes 
via  the  Service  Component  Acquisition  Executives  (CAEs)  and  DoD 

•  Create  an  acquisition  Management  Flight  Simulator 

-  Educate  program  staff  as  to  pitfalls  awaiting  them,  as  well  as  potential  mitigations 
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